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Abstract — In this work, we consider pseudocodewords of (re- 
laxed) linear programming (LP) decoding of 3-dimensional turbo 
codes (3D-TCs), recently introduced by Berrou et ah. Here, we 
consider binary 3D-TCs while the original work of Berrou et al. 
considered double-binary codes. We present a relaxed LP decoder 
for 3D-TCs, which is an adaptation of the relaxed LP decoder 
for conventional turbo codes proposed by Feldman in his thesis. 
The vertices of this relaxed polytope are the pseudocodewords. 
We show that the support set of any pseudocodeword is a 
stopping set of iterative decoding of 3D-TCs using maximum 
a posteriori constituent decoders on the binary erasure channel. 
Furthermore, we present a numerical study of small block length 
3D-TCs, which shows that typically the minimum pseudoweight 
(on the additive white Gaussian noise (AWGN) channel) is smaller 
than both the minimum distance and the stopping distance. In 
particular, we performed an exhaustive search over all interleaver 
pairs in the 3D-TC based on quadratic permutation polynomials 
over integer rings with a quadratic inverse. The search shows that 
the best minimum AWGN pseudoweight is significantly smaller 
than the best minimum distance. 

I. Introduction 

Turbo codes (TCs) have gained considerable attention since 
their introduction by Berrou et al in 1993 [1] due to their 
near-capacity performance and low decoding complexity. The 
conventional TC is a parallel concatenation of two identical 
recursive systematic convolutional encoders separated by a 
pseudorandom interleaver. To improve the performance of 
TCs in the error floor region, hybrid concatenated codes 
(HCCs) can be used. In [2], a powerful HCC nicknamed 3- 
dimensional turbo code (3D-TC) was introduced. The coding 
scheme consists of a conventional turbo encoder and a patch, 
where a fraction A of the parity bits from the turbo encoder 
are post-encoded by a third rate-1 encoder. The value of A 
can be used to trade-off performance in the waterfall region 
with performance in the error floor region. As shown in [2], 
this coding scheme is able to provide very low error rates for 
a wide range of block lengths and code rates at the expense 
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of a small increase in decoding complexity with respect to 
conventional TCs. In a very recent work [3], an in-depth 
performance analysis of 3D-TCs was conducted. Stopping sets 
for 3D-TCs were treated in [4,5]. 

Feldman et al. [6] recently considered the use of linear 
programming (LP) to decode binary linear codes. The obvious 
polytope for LP decoding is the convex hull of all code- 
words, in which case LP decoding is equivalent to maximum- 
likelihood (ML) decoding. However, in general, the convex 
hull has a description complexity that is exponential in the 
codeword length. Thus, Feldman et al. [6] proposed a relaxed 
polytope which contains all valid codewords as vertices, 
but also additional noncodeword vertices. The vertices of 
the relaxed polytope are basically what the authors called 
pseudocodewords in [6]. 

Relaxed LP decoding of conventional TCs was first de- 
scribed by Feldman in his thesis [7]. In [8], it was shown 
that the set of points from the relaxed polytope (as described 
by Feldman in [7]) where all entries are rational numbers is 
equal to the set of all pseudocodewords of all finite graph 
covers of the TC factor graph. Also, a many-to-one correspon- 
dence between pseudocodewords and stopping sets for turbo 
decoding on the binary erasure channel (BEC), as originally 
defined in [9], was observed. The many-to-one correspondence 
can be described as follows. The support set of any (graph- 
cover) pseudocodeword, i.e., the set of nonzero coordinates, is 
a stopping set, as defined in [9], and for any stopping set there 
is a pseudocodeword with support set equal to the stopping set. 

Expressions for the pseudoweight on the BEC, the binary 
symmetric channel (BSC), and the additive white Gaussian 
noise (AWGN) channel are given in [10,11]. The pseu- 
doweight is the equivalent of the Hamming weight on the 
BEC and the BSC, and the equivalent of the squared Euclidean 
distance on the AWGN channel, when LP decoding and not 
ML decoding is performed. Computing the exact value of the 
minimum pseudoweight on any channel (possibly besides the 
BEC) is very hard, and the exact value is not known even 
for the well-known (155, 64, 20) Tanner code from [12]. The 
best upper bound is 16.4037, found using the search algorithm 
from [13]. 
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Fig. 1. 3D turbo encoder. A fraction A of the parity bits from both constituent 
encoders C& and Cb are grouped by a parallel/serial multiplexer, permuted 
by interleaver n c , and encoded by the rate-1 post-encoder C c . 



In this work, we adapt the definition of a pseudocodeword 
for a conventional TC from [8] to 3D-TCs. Furthermore, 
we show by several examples and by computer search that 
typically the minimum AWGN pseudoweight, denoted by 
^mh^° N ' * s smaller than both the minimum distance d m i n and 
the stopping distance h m { n for these codes. 

II. Coding Scheme 

A block diagram of the 3D-TC is depicted in Fig. 1. The 
information data sequence u of length K bits is encoded 
by a binary conventional turbo encoder. By a conventional 
turbo encoder we mean the parallel concatenation of two 
identical rate-1 recursive convolutional encoders, denoted by 
C a and Cb, respectively. Here C a and Cb are 8-state recursive 
convolutional encoders with generator polynomial g(D) = 
(1 + D + D 3 )/(l + D 2 + D 3 ), i.e., the 8-state constituent 
encoder specified in the 3 GPP/UMTS standard [14]. The 
code sequences of C a and Cb are denoted by x a and Xb, 
respectively. We also denote by x TC the codeword obtained 
by alternating bits from x a and xt>. A fraction A (0 < A < 1), 
called the permeability rate, of the parity bits from x TC are 
permuted by interleaver II C (of length 7V C = 2XK), and 
encoded by an encoder of unity rate C c , called the patch 
or the post-encoder [2]. This can be properly represented 
by a puncturing pattern p applied to x TC (see Fig. 1) of 
period 7V P containing A7V P ones (where a one means that 
the bit is not punctured). The fraction 1 — A of parity bits 
which are not encoded by C c is sent directly to the channel. 
Equivalently, this can be represented by a puncturing pattern 
p, the complement of p. We denote by x c the code sequence 
of C c . Also, we denote by x£ h and x£ h the sub-codewords of 
x a and Xb, respectively, sent directly to the channel, and by 
x ch the codeword obtained by alternating bits from x^ h and 
x£ h . Likewise, we denote by xg and x£ the sub-codewords 
of x a and x b , respectively, encoded by C c , and by x p the 
codeword obtained by alternating bits from xg and x£. Finally, 
the information sequence and the code sequences x ch and x c 
are multiplexed to form the code sequence x, of length TV bits, 
transmitted to the channel. Note that the overall nominal code 
rate of the 3D-TC is R = K/N = 1/3, the same as for the 
conventional TC without the patch. Higher code rates can be 
obtained either by puncturing x ch or by puncturing the output 
of the patch, x c . 



In [2], regular puncturing patterns of period 2/ A were 
considered for p. For instance, if A = 1/4, every fourth bit 
from each of the encoders of the outer TC are encoded by 
encoder C c . The remaining bits are sent directly to the channel, 
and it follows that p = [11000000] and p = [00111111]. 

III. Stopping Sets for 3D-TCs 

In this section, we review the definition of stopping sets for 
3D-TCs from [4,5]. First, we need to define some concepts 
related to the support set of a vector and of a set of vectors. 

The support set %(x) of a binary vector x = (#o, • • • , xn-i) 
(of length N) is the set of nonzero coordinates. As an example, 
with x = (0,1,1,0,1), x( x ) = {1,2,4}. The support set 
x(C) of a binary linear code C is the union of the support 
sets of each codeword in C, and the support vector of C, 
denoted by "0(C), is the unique binary vector whose support 
set is equal to x(C). In the following, the notation C _1 (c) 
with a binary vector c will denote the output of a maximum a 
posteriori decoder for code C when decoding c on the BEC 
where the 1- symbols in c are treated as erasures. 

We start by reviewing the definition of a stopping set for 
a conventional TC, i.e., the 3D-TC without the patch. The 
following definition is taken from [9]. 

Definition 1 ([9]). Let C denote a given conventional TC 
with interleaver II. A subset S = 5(11) = (<S U , <S Xa , <S Xb ) ^ 
{0, . . . , N — 1} of the coordinates of the output sequence 
(or codeword), where <S U , <S Xa , and <S Xb are subsets of the 
codeword coordinates corresponding to systematic bits, parity 
bits from C a , and parity bits from Cb, respectively, is a 
stopping set if and only if the following conditions are satisfied. 

1) There exist linear subcodes C a C C a and Cb C Cb, 
such that /i a (x(C a )) = <S Xa and /ib(x(<^b)) = Sx b , 

2) n(x(C- 1 (^(C a )))) =x(C b - 1 (^(C b ))), and 

3) / i s (x(C- 1 WC a ))))=5 u . 

/i x (0 is the mapping of indices of parity bits from C x to indices 
of the turbo codeword, x = a, b, and /i s (*) is the mapping of 
indices of systematic bits to indices of the turbo codeword. 

The definition above can be generalized to 3D-TCs in a 
fairly straightforward manner. The following definition is taken 
from [5]. 

Definition 2 ([5]). Let C denote a given 3D-TC with 
interleavers H and H c . A set S = <S(II,II C ) = 
(<S U , <S x ch , <S x ch , <S Xc ) C {0, . . . , iV — 1} of the coordinates of 
the output sequence (or codeword), where S u , <S x ch, <S x ch, and 
<S Xc are subsets of the codeword coordinates corresponding to 
systematic bits, parity bits from C a sent directly to the channel, 
parity bits from Cb sent directly to the channel, and parity 
bits from C c , respectively, is a stopping set if and only if the 
following conditions are satisfied. 

1) There exists a linear subcode C c C C c , such that 
Vc(x(C c )) = S Xc , and 

2) (<S u ,<$ x p U <S x ch,<S x p U <S x ch) is a stopping set of the 
outer TC, as defined in Definition 1, where <S x p is the 



set of indices from Ii c 1 (x(C c 1 (iJj(C c )))) corresponding 

to parity bits from C x , x = a, b. 
/i c (0 is the mapping of indices of bits from the output of the 
inner encoder to indices of codewords of the 3D-TC. 

The size of a stopping set 5 is its cardinality, and the size 
of the smallest nonempty stopping set is the stopping distance 



where 



IV. LP Decoding of 3D-TCs 



In this section, we consider relaxed LP decoding of 3D- 
TCs, adapting the relaxation proposed in [7] for conventional 
TCs to 3D-TCs. 

Let T x = T X (V X ,E X ) denote the information bit-oriented 
trellis of C x , x = a, b, c, where the vertex set V* partitions 
as K = u£l Vx,t, which also induces the partition = 
u£l0 1 £ , x,t of the edge set £? x , where 7 X is the trellis length of 
T x . In the following, the encoders C x (and their corresponding 
trellises T x ) are assumed (with some abuse of notation) to be 
systematic, in the sense that the output bits are prefixed with 
the input bits. Thus, C x is regarded as a rate- 1/2 encoder, 
and the trellis T x has an output label containing two bits, for 
x = a, b, c. Now, let e G E^ t be an arbitrary edge from the tth 
trellis section. The zth bit in the output label of e is denoted 
by Ci(e), i = 0, . . . , n Xjt — 1, the starting state of e as s s (e), 
and the ending state of e as s E (e). Define the polytope 

Q x = {(y,f)e[0,l] N+2AK x[0,l]l^: 

fe= Y] fe, VV G V x ,t, 
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eeE x , t :Ci(e) = l 



1, and 



/c * = 0,. 



0, . . . ,n x ,t - 1} 



(1) 



where x (t, i) = Sj=o n x,j+^ and p x (*) denotes the mapping 
of codeword indices of constituent encoder C x to codeword 
indices of the overall codeword of the 3D-TC appended with 
the 2XK parity bits from encoders C a and Cb which are sent 
to the patch. 

Next, define the following projection onto the first TV 
variables of y 

Q x ={ye [0, 1} N : 3 f G [0, y G [0, 1] 2AK , and 

((y,y),f)e<2x} 

and the polytope 

<2n,n c = Q a n Q b n Sc. 

Relaxed LP decoding (on a binary-input memoryless channel) 
of 3D-TCs can be described by the linear program 



N-l 



minimize Xiyi subject to y G Qn,n c 



(2) 



Xi log 



Pr{r z | Q =0} 



Z = JV — 1, 



^Pr{r z | Q = l} / 

q is the /th codeword bit, and is the /th component of the 
received vector. If instead of Qn,n c we use the convex hull of 
the codewords of the 3D-TC, then solving the linear program 
in (2) is equivalent to ML decoding. 

The notion of a proper and C -symmetric polytope was intro- 
duced in [7, Ch. 4] where the author proved that the probability 
of error of LP decoding is independent of the transmitted 
codeword on a binary-input output- symmetric memoryless 
channel when the underlying code is linear and the polytope 
is proper and C- symmetric. 

Lemma 1. The polytope Qn,n c is proper, i.e., Qn,n c H 
{0, 1}^ = C and C- symmetric, i.e., for any y G Qn,n c cmd 
c G C it holds that |y — c| G Qn,n c - 

Lemma 1 can be proved using the same arguments as for a 
conventional TC. The formal proof is omitted for brevity. 

When decoding 3D-TCs by solving the linear program in 
(2), a vertex point of the polytope Qn,n c is always returned by 
the decoder. Furthermore, any vertex point is a pseudocode- 
word, and the pseudoweight on the AWGN channel of a 
nonzero pseudocodeword uj is defined as [10, 11] 
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We can prove the following lemma. 

Lemma 2. Let C denote a given 3D-TC with interleavers II 
and II C . For any pseudocodeword u), the support set x(^) °f 
u) is a stopping set according to Definition 2. Conversely, for 
any stopping set S = 5(11, II C ) of the 3D-TC there exists a 
pseudocodeword uj with support set x(c*;) =5. 

As a consequence of Lemma 2, it follows that w^ GN of 
C is upper-bounded by the h m [ n of C. 

We remark that the d m i n can be computed exactly by solving 
the (integer) program in (2) with A/ = 1 for all I, with integer 
constraints on all the flow variables f in (1), i.e., // G {0, 1} for 
all I, and with the constraint E^o 1 Vi — 1 t0 av °id obtaining 
the all-zero codeword. The exact d m { n of 3D-TCs has not 
been computed before in the literature. For instance, in [3], 
only estimates of the d m i n were provided. Finally, note that 
the exact h m { n can be computed in a similar manner using 
extended trellis modules for T x (see [9] for details). 

V. Numerical Results 

In this section, we present some numerical results when 
the interleaver pair (II, II C ) is taken from the set of all 
possible interleaver pairs, and when it is taken from the set 
of pairs of quadratic permutation polynomials (QPPs) over 
integer rings. Permutation polynomial based interleavers over 
integer rings for conventional TCs were first proposed by Sun 
and Takeshita in [15]. These interleavers are fully algebraic 
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Fig. 2. Estimated w^in > wsmg £/ze algorithm from [13], adapted to 
3D-TCs, and exact d m i n for 3D-TCs with 100 randomly selected pairs of 
interleavers (green plus signs) and with 100 randomly selected pairs of QPP - 
based interleavers (blue x-marks). K = 128 and R = 1/3. 



and maximum contention- free [16], which makes them very 
suitable for parallel implementation in the turbo decoder. QPP- 
based interleavers for conventional TCs were recently adopted 
for the 3 GPP LTE standard [17]. Finally, we remark that for all 
the results below, A = 1/4 and the regular puncturing pattern 
p = [11000000] are assumed. As shown in [3], A = 1/4 gives 
a suitable trade-off between performance in the waterfall and 
error floor regions. 

A. Ensemble-Average Results for K = 128 and R — 1/3 
In Fig. 2, we present the exact d m i n and an estimate of 



w: 



£WGN denoted by 



w 



AWGN 



, of unpunctured 3D-TCs with 
K = 128 and with 100 randomly selected pairs of interleavers 
(II, II C ) (green plus signs). The corresponding results with 
QPP-based interleaver pairs (and with no constraints on the 
inverse polynomials) are also displayed (blue x-marks). For 
all codes, except 11, the estimated w m ^ GN is at most equal 
to the d m in. The minimum AWGN pseudoweight w m ^ GN 
was estimated using the algorithm from [13], adapted to 3D- 
TCs, with an SNR of 2.0 dB and 500 evaluations of the 
algorithm, while the d m i n was computed exactly by solving 
(2), augmented with the constraint X^o* Vi ^ 1 ( to avoid 
obtaining the all-zero codeword), with integer constraints on 
the flow variables f in (1). Note that when the d m i n is strictly 
smaller than the estimated w m ^ GN , the estimation algorithm 
from [13] was unsuccessful, since w m ^ GN is upper-bounded 
by the d m i n (see Lemma 2). From the figure, we observe that 
QPPs give in general better codes, and that ^m^ G ^ is strictly 
lower than the d m i n for most codes when the d m i n is large. 



AWGN 
min 



B. Exhaustive/Random Search Optimizing w\ 

In this subsection, we present the results of a computer 
search for pairs of QPPs with a quadratic inverse for K — 128 
and 256 for unpunctured R = 1/3 3D-TCs. The objective 
of the search was to find pairs of QPPs giving a large 
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AWGN 



estimated using the algorithm from [13], adapted 
to 3D-TCs, with an SNR of 1.7 dB and 500 evaluations of 
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Fig. 3. Exact d m [ n (red circles), exact h m in (green x-marks), and w: 



(blue plus signs) of the 75 best (in terms ofw^ 



AWGN 
min 



^ ) QPP-based interleaver 



pairs for the 3D-TC with input block length K = 128 and code rate R = 1/3. 



the algorithm. To speed up the search, an adaptive threshold 



AWGN 



was set in 



on the minimum AWGN pseudoweight w 
the search, in the sense that if a pseudocodeword of AWGN 
pseudoweight smaller than the threshold was found, then this 
particular candidate pair of QPPs was rejected. 

For K = 128, we performed an exhaustive search over the 
entire class of pairs of QPPs (with a quadratic inverse). In 
Fig. 3, we plot the exact d m [ n (red circles), the exact h m [ n 
(green x-marks), and w m ^ GN (blue plus signs) of the 75 3D- 
TCs with the best w^[ GN . For each point in the figure, the 
^-coordinate is the sample index (the results are ordered by 
increasing dmin), while the ^/-coordinate is either the exact 
dmin, the exact h m [ n , or ^m^ GN . From the figure, we observe 
that the best ^^ GN (which is at most 30.2139), is strictly 
smaller than the best possible d m [ n /h m [ n . The best possible 
dmin was established to be 38 (exhaustive search), and for this 
particular code the exact h m [ n is 36. 

For K — 256, only a partial search has been conducted. The 
largest value for ^n^ GN found in the search is 42.9650 after 
taking approximately 100000 samples from the search space. 

C. Exhaustive/Random Search Optimizing d m i n 

We also performed an exhaustive/random search for pairs 
of QPPs with a quadratic inverse for selected values of K for 
unpunctured R = 1/3 3D-TCs. For K = 128, 160, 192, and 
208, the search was exhaustive, in the sense that each pair 
of interleavers was looked at. The objective function in the 
search was the d m i n , which was estimated using the triple 
impulse method [18]. The results are given in Table I for 
selected values of K, where f(x) = fix + f2X 2 (mod K) 
generates the TC interleaver, f(x) — f\x + f2X 2 (mod N c ) 
generates the permutation in the patch, and d m in, d m ? n , and 
^mh^ GN denote the estimated d m in, a lower bound on the d m in 
based on an ensemble analysis, and the estimated ^^ WGN 



AWGN 



was 



respectively. The minimum AWGN pseudoweight w 
estimated using the algorithm from [13], adapted to 3D-TCs, 
with SNR and number of evaluations of the algorithm given 
in the tenth and the eleventh column of the table, respectively. 



TABLE I 

Results from an exhaustive/random search for pairs of QPPs with A = 1/4, both with a quadratic inverse, in which the first QPP 

f(x) GENERATES THE TC INTERLEAVER AND THE SECOND QPP f(x) GENERATES THE PERMUTATION IN THE PATCH. 



K 


fi 


h 


N c 


fi 


h 


^min 


a min 


~ AWGN 
min 


SNR 


Evaluations 


128 a 


55 


96 


64 


9 


16 


38^ 


19 


29.6042 


2.0 dB 


2000 


160 a 


131 


60 


80 


9 


20 


42 b 


20 


30.0000 


1.7 dB 


500 


192 a 


35 


24 


96 


11 


12 


46 


22 


32.9046 


1.7 dB 


500 


208 a 


165 


182 


104 


37 


26 


49 


22 


36.3370 


1.7 dB 


500 


256 


239 


192 


128 


37 


32 


52 


24 


42.7816 


1.7 dB 


500 


320 


183 


280 


160 


57 


20 


58 


26 


41.3818 


1.7 dB 


500 


512 c 


175 


192 


256 


15 


192 


67 


33 


45.5872 


2.0 dB 


500 



a Exhaustive search. 

b This is the exact d m - in . 

c The QPPs are taken from [3]. 



Note that the number of evaluations of the algorithm indicates 
its reliability. The lower bound on the d min given in the eight 
column of the table should be interpreted in the following 
manner. When picking random pairs of interleavers from the 
3D-TC ensemble, the d m i n will be at least equal to the value 
of the bound with probability 1/2. Finally, we remark that the 
codes in the first and second rows, for K = 128 and 160, are 
^min-optimal, in the sense that there does not exist any pair of 
QPPs (with a quadratic inverse) giving a d m { n strictly larger 
than 38 and 42, respectively, for the unpunctured 3D-TC. 

D. K = 512 and Different Code Rates 

In [3], a pair of QPPs for K = 512 giving an estimated 
d min of 67 for R = 1/3 was tabulated in Table IV. Also, 
the d m i n with optimized puncturing patterns, giving estimated 
minimum distances of 38, 17, and 15, for nominal code rates 
R = 1/2, 2/3, and 4/5, respectively, was tabulated. By 
using the algorithm from [13], adapted to 3D-TCs, we found 
pseudocodewords of AWGN pseudoweights 45.5872, 28.8257, 
19.1668, and 10.1984, for nominal code rates 1/3, 1/2, 2/3, 
and 4/5, respectively. 

VI. Conclusion 

In this work, we considered pseudocodewords of (relaxed) 
LP decoding of 3D-TCs, generalizing the LP relaxation pro- 
posed by Feldman in his thesis for conventional TCs. We 
showed that the support set of any pseudocodeword is a 
stopping set, and presented a numerical study of small block 
length 3D-TCs, which showed that typically w^ GN * s smaller 
than both the d m in and the h m in for these codes. It is expected 
that the w^ GN will dominate performance for high SNRs. 
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